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TO: Board of Higher Education 

Academic Affairs Conn^^ttee 

FROM: Chancellor T. £^3 
SUBJECT: College Outcome/ E^vTl^t ion Program 

INTRODUCTION 

develo'pment orrcompr'eh^*si°e 'orocrJ'V T'^"^^^ ''''' °n the 
higher education. It waf Igreed ?he^ Sat''?^'V ^^^^^"^""'es of 
program has merit and that the next steo c^ouih ht ^k^"""* °^ ^ 
a proposal for your consideratinn Tht'^n ^ ^^^^^ presentation of 
to outline the principles aJidP^ °^ ^^'^ memorandum is 

we should folloE "n devIloD?na In^h'; n""^ course of action I believe 
concur. aeveioping such a program, if you and the Board 

and SLe"o7ner;7til^^^^^^^ listing practices 

would fulfill two broad goals Fir.^ s^- ^ outcomes, the program 
efforts by faculties, admfnistmorr^nd Vh"^1 ^V^^'' stimulate 
enhance curricula, studmt Wn^^^^^^ students to evaluate and 
performance. Secondly; it wol^d l^L^..o^^ institutional 
addressed to the concerns anHe^an^r.^ n^^^ reliable data 

the integrity of degrees and the value ''h'' '° 

program should achieve it.: aimc u collegiate education. The 

evafuation activU L'f facn^•StedTv^^nrpntf campus-based 

central ly-administered ellVull^TM^^^^^^^^ 

is res?onsi>e\Vi-s"^uef raTs^ evaluation program 

National Institute of Education R^no^^^^^^ ""^^^ ''e^"* 

For this reason I believe ft cn.fnHi^' "Involvement in Learning." 

cations are Poten??anr fi--reachTnn^ ?n°""^'^' r^e""^^ i-^Pli- 
prudent to proceed with care 'I?' however, I believe, it only 

sensitive to the dliersitv of ?hp '^rSSI^ ?'^ • """^^^ we are 

multitude Of conceXl^'?;sres^^cVr^eVoi;Lr^"^^^^^^^^^^ 
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outlined below and I «Ji, Ji..- ^ Progran preliminarily 

course of action to the Boarl. ^"^ ""'•""'•ence in reconraending thij 

roiMClPLES 

all InllSSJUl "actVvniVs" 'TT.J' ^n""^' ■i'''"'"^ "'«' °^ 
articulate goals and standards^f per "r~^^^ 

Which these goals are mpf anH u * *"est to the degree to 

performance ^Asslssment^V also' ^ nfrJ^T'' ^^^^back. enhance 
competence and distributa reward^ a "^^f'^'^^J * '"^^"S to Judge 
inherent to the prowls ThfrX^r^^^ t.. ^^9^^^ °^ tension is 

and Its 4lteriidr2^ d^^^^^ 

gram. Cur initiative should be so structurpS lc^\ * """.^ P^°- 

«is^uSd 'ii's^x-t' r ;?^Hs4H"? t" ^ 

educational process. - exte»nal intrusions into the 

aspec'J%?l?is''ien;?in' 'loVthe;' tt^' 'j^'i^^' '''''''''' *° -^c^' 
coSinittee in carefully we i ah inn th'p nnH ^'^^^^^ ^^^'^^ advisory 
ments of the prograS. potentially conflicting require- 

1. Maintaining Publ ic Confidence 

^^^^^^^Terence to Ucarly dermPd ang-T-abTTcTy- S^tj cu I ated 
2- Nurturing Institutional Autono my and Individu;n nsw...,-^ 

tolerated hut a virtue to be rhlrihlx X * ""usance to be 
designed and utiu Z in wave ^Mrh Program should bP 

mst?tutional and .rlT^ll.r^V^^^ 
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3. Stimulating Educational Excellence 

««nrr'^ihf^"*' Objectively demonstrating student and institu- 
tional achievements and nurturlna d1ver«:itv co* +hr\« j • 
the program but should not wStlse Its ess^^^^ ^°nnn^'*'? *° 
tlonal considerations should ultllalely provide the b!s?s fnr"f?7 
?nS5'?5 f ?5 The program^ sZ f . i.^i^ TJJ^L ^11 

LhruMj T^. J T^cin jy _and admini strators d^^^ np 



GUIDELINES 



Whereas the principles outlined ahowp oc+ahHeh - ,1 

for the progr»/the guSdelines ?hat ?onow sho d ,„?°^^^^ 
con™ittee-s specific reco«endat1ons These ouldeMnTs nrnJ^Ii"? 
'""h J"*"* Task Force on Pre-Colle^e P^^^^^^^^^ 

rthe--u. Tth? hW'» H HFerF-""- 

undergraduate curricula. reliable data on the effectiveness of 



1 . Content and Measur ement 
a. 



b. 



The intellectual development of individual students i.; thp 
primary function of all institutions' of higher educatioS 
Thus improving the content and reliability of informat^f^^^^ 
this area, on an individual basis, should be a cenS?a task 
aL .xiS""- J^'' '"^'^ ^^"11 "-^^^'^^ ^ comb1nat?on of new 
gS?ne^ cJl"?H ' k'*"™1"H °^ assessment. The information I J 
Issist c5°HInf.c^^• "'^^ to evaluate student performance, to 
rlnifZ. JV^"*! I'nProving their skilU, to guide their 
?nH °/ ^"^^J^ progrms and careers, to inform curricu- 
IS^tMTtaa. *° ^^^^"3*^^" Public""con^^^5;n%"e 

A second task of the program should be to produce reliable 
bf thP ,-n??>3t institutional outcomes in areas" dent ?ied 
by the institutions as relevant to their missions and their 
special circumstances. This involves evaluation nf^t^I 
impact institutions have on theiY studPntc L 
community-based groups an°d" i'nsmut?Snsr%n" socTetylwiSe 
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c. 



tipns beyond the fo,^^ in^^^^ol'lf',"^^^^;:}^"'^' ""f'^"- 

In order to determine the contributions Institutions n»i.. 
ih1rever'?ossi?l.'°*"T."^'"^,;""'« sto 5"* e ap™ ed 



2. Standards 



JSn?eV*?^"^to'u/;yt.ir^^^^ s^o^^bVctiref-v;^^ 

define acceptable leveu noZ^P«-Jr, u ^"^s* ""S to 

institutions The cfrnnH P^r*^°r'"a"ce by both students and 

to improve ihei> p|rfo?L^^^^ ""^ institutions 

respec't to sta^^^^-drrnntfonaf no^^^ °^ -'t^' 

3. Locus of Responsibility 

The program's dual purposes (see naop n w-n 

ir?»ed3s9rr^^'-'"'^^^^ 

Skiffs P%gram h«'USoht „s''^°h^T "Pe-:'*"" of the bSs?J 

agdemic coUnit/. c»lirres\tcrsrcan^";iW^ 

a. While lines of demarcation cannot be neativ Hr.au,n 4.u 

public confidencP hn'c* \ requirement to maintain 

?entral,yr;e1rprd ,„^d^ locl^U^elop^^d 4asur:"'^°" °^ 



Srar*?;fe"!n\"?Sra?w!;as''e?''con'?t \''^ °^ ^""^'"^^O" 

=rti;; 4^ra,-and";SS .^r£H^\^^^^ 
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b. With reference to the individual performance of studpntc +ho 
Department's role should be to assist iL?i-t.if?nnc' ?® 

4. Confidentiality 

Institutions and the Department should use the data and nthp^ 
r^iSt" IVl'fiied°"hl' t'^ way's'^whi-ch' co^'fo^ S 

eSablishe^^^ nrar??^I t "^.u**'^ °^ "^'9^'^^ Education, and to 

SrS w*^" " ^^^^^ 

5. Costs 

IS difficult to estimate actual costs with any degrerof TrlcisioiJ 
battery 'f^'e 'usTis rouiL'''''' *° ' cen'traffylde/ig'n'ed 1 ? 
CollS^* Bl'sir Skills" Pla?^^^t°;;sl?\^JsTm?;^^^^ '^a^l 1^1^ t1,Tn"^ 
im'SfLpnU-'^^^'??.^. *° 51.500.000 for develoS^Int and first-year 
nn!r^l!-!-°"-, °^ evaluation and data analyses mfght 

incur additional costs of equal magnitude. The Stat^ thrnf.nh ?hn 
Department, should properly' bear ^the costs for those a'spec\^^ 
the program which are centrally controlled and mandated. 

Financial incentives hold considerable promise for 5timiii;,+inn Sm 

?^on'ar?„tnJ" ^""i^i^tion^l performance' ani'^ncouragrg iJs?^ 
tional involvement and cooperation in all asoects of thS nrnllll 
The Department should. make full use of th'^s Dol[cy oDt?on^; 
ensure the program's success. poncy option to 
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IMPLEMENTATION OPTIONS 



Beyond these general principles and quidelines a hnc+ 

^' Options for Development of Performance Measures 

"outcomes" of higher education can be clustered in thp 
tSJl? .rL''*^"-. ^^l performance, as a n^asure of ^nd^j! 

Ittll ^nclf.T"* Institutional outcome; (b) commu^iiy. 

based institutional outcomes; and (c) society^ide institutional 

?li eTof th'eslTet'"""? P-gr'rifould'cLver'^ 

^reas. In each case, it will be necessary to 
choose among a broad spectrum of possible indicat??s «nri tn 
determine the appropriateness of each indicator select^S 
nlllT^^^K''^ principles of mainJaining pub"c wn'id^ 
ex^en;n?e/"'°"°"^ diversity, and stimll^atini' educl2?onal 

nf +h!^lV"/*i.°"* °^ intellectual development must be at the center 
of the effort to assess student performance In thf^ ron^.n 
ZI:T^ distinction should be drawn between the acqu?s???o^ o? 
?r.no!.,-^;"^3l"^?^ ""^^^'^ proficiencies and the acquisTtion 
d1scl?s^/^\hfr-^^^^^^^^ 

?.^^TK^^^^^^ ^:d%hrsfcon5 

,-nH,- ^°*!?^'^ aspect of student performance, relevant for both 
cJ^Jon?;'^- '"^ institutional evaluation, invo ves the Access" S 
fj" * ^^S^'e^ and- the post-graduation activities 

-P ogts's' t'Jvfar'; the 'd1cr°ee V'}^'' These"may ]n;?;dl 

ele^?t.Z^' e^cat^r e;i7^.\"n\\°" ' ^^ourT in" 
performance in these areas, care ?ust be takS to account ^orJS^ 
and the1tate''n;':f;'"'^"°"'' ^"^^ ^^uden? motivation 

d?scu«\/Vh'ese°iss!;ts!""°"'* ' ^P^^es 12-15) further 

arpa l^® above comnents suggest the following components in the 
Z^W^ 'J"^^"* outcomes, other components may Si so be appro! 
priate. and may be added by the advisory committee. Some of these 

o K'«nnlJ°" o i'J'^'nd"''""^^'^^ comp^lile'Tatf S?? 

^JIT^ ^t^t^, VmVov"ii'eit\° ^''^^'^ 
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a test battery that measures proficiencies in writina 
quantitative reasoning, critical thinking.! and any 
areas appropriate for the evaluation of general college-levll 
academic -proficiencies, to be taken toSard the end of the 
sophomore year by all students. The same instruments should 

?[ll sM,^fd"? rodents ' proficiencies In the c 
skills compared to their proficiencies at college entrance 
as originally measured by the New Jersey College Basic Skills 
Placement Test. The test battery may Include some coLonentc 
common to all Institutions and others specific ?o in7iZul' 
Institutions or groups of Institutions; inaiviauai 

llnll'''}°'^^^^^^^ specific graduation require- 



ments. including satisfactory pVr^oVm^nc'e Tn° omprehln 
"ITd"oTc°SSceSUlonf ^" '^'''^ 



ve 
major 



— development of measurements of progress toward the dearee anH 
post-graduatlon activity relevant to institutioJS? 
assessment; particular attention should be paid to mino?i?v 

SlJ^SJiaScef"' '''''''''' ' "^^^"^^ o7^'insti?;;ionaf 

^iflnT??rSn? t'^""*Vi'5 Students, collegiate institutions provide 
significant benefits to other qrouos Fval..ati-nn *ul 
outcomes Should be included in\ co'm^rehVnsivl'pro^^^^^^ JSnll 

ltJces''\^:r^^^^^^ ^^^^"ds' oTTocal ciS! 

stances. They might Include employers, participants in adult 
education and cultural activities, focal governments and nolicv 
:e?ra?e°rg^'c^-;r^ educationa'l°7ns"?ft"utions'. s'oJjaf 

Society-wide outcomes include all the manifold ways in which 

J^n. ''•r.^""!^ ^"^ resources of institutions of hi g?l? educat Sj 

?ScfuHpf %hp° H°-"^ '•^S^ °^ society-li^e outwmes 

includes the discovery and development of new knowlpdop ThI 

application of knowledge to solve teSlnical and soci'etal ^r^^^^^^ 
heJitaae '^"Thn'Tn-" ^"^ interpretation of our diverse cSltSTIl 
s?ho?a??; .nH ./""^'T^' °"tco"'es include the 

lSil?w«™nnftl^*'''^^ policymakers, corporations, the 

artistic community, and the entire body politic. 



1 This category includes the ability to identify and propose wavs of 
splving problems the ability to draw reasonable cSS?l SI io?s from 
rtSrjri(p\"glsrO-n).^^^'^^^ ^"^^ - those 1Js°t"ld'^?^ 
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iH»„tI5* folloKing process should be Instituted in order to 
Jt'Sel^sTevttthr "'^"^-1* out?q4'r ani 

" Hn*e1'j|?"?i;*-evVluJ??o„"onhi\'\'utcr™es'; ""^^^ '"^ '"'^^ 

— selection, at the local level, of appropriate outcomes ;«nd 
development of instruments and practices for their evalS- 

— identification of models for the sound and obiectivp 
evaluation of these outcomes and development orjoli-cfes for 
their use on a systemwide basis. Huncies ror 

2- Options for Feedback and Information Sharing 

Information systems presently in use should be reviewed and 
If necessary, redesigned so as to report the results of ^!?rnJn: 
assessments in ways that can be used constructively A reSesS 
l^Ir'n"na''°a^d "^^""^ data'^f^wa^' thai'1 S 

InlTronLnt .jfnL,^^^^^^^ P'-QSrams. institutional 

environment, and institutional performance. It should facilitatp 

n%'?n;?3SSs' serU".ihrj'"T''*^'°" ^^ween receiving and i d nl 
insxitutions (see Attachment I, pages 14-15). Final! v it chnniH 

yield information that is meaningful and accessible to' tJe pSbl c 
and to organizations interested in higher education. ^ 

The Department's current data systems are confinpH <•« 
collection of institutional data in tl^e aggregate T^^^^^ ha? 
proved inadequate for studies of ,udent" outcomes iuch II 
retention and progress toward the deoree Data hnanSnn tl 
post-graduation activity have never beerc?llected \en\^\n^ °X 
c?ln7"''^? '-T^' ^^"'"S"^^ *° P'-ovide information " ind' iVdual 
student characteristics, persistence, transfer, and w thd?iwal 

neces a?S -^n^'^nt^i '/^^'V- ''"-'^^'^ prov?de the Sati 

necessary for both information sharing and assessment ^i.rh I 

ftZV.'"^""' """''^ "a-dardlze? def1n?"ons7n5' reporting 



RECOMMENDATIONS 
1. 



A comprehensive program to evaluate the outcomes of colleaiate 

?hrDotentiar^'' -J^ -'^'"^"^ implemen?ed in detail .' has 
the potential to contribute significantly to the achievement of 
educational excellence in New Jersey, i therefore ask vSu to 

S^?7n °' "^'9^^^ """^^'^^ thaTu authorize the 

Chancellor to institute such a program. 
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• 2. In order to attain optimal results, the program requires the in- 
formed advice and participation of the academic Sai^Snity. 
IhH lo"?^ proceed on a sound footing. I ask you to recommend 
that the Board appoint an advisory corrmittee upon the Chancellor's 
nomination. The committee should be composed of know! edaeabl e 

members of the public who can reflect the views of various con- 
stituencies of higher education such as business, government a^d 
non-profit organizations. It should also include iSdtviZu Iho 
have the necessary technical expertise to evaluate and adv se on 
all aspects of the program. ciavise on 

3. I ask that you recommend to the Board a charge to the coimittpp 

(pa5es'"7'8f .1^5?'"^'^ °^ evaluat'ion outlineHbSvI 

iS!??!*,! ^" committee should study the 

?n in?t^l.J£*^°"' and report to me its recommendations on how bes? 
J?nho.' comprehensive system of evaluating the outcomes of 

higher education. The committee should further lonsider. in eaSh 
case, the information systems required to facilitate data 
collection feedback and information sharing, the incentives 

!?^"Jr^^. Pe-^formance. where necessary', and the opj on" 

for bearing the costs of the program. uh«.'u"5 

CONCLUSION 

tlnn^^nnrrr,-c*M* t^^Ts Proposal for a comprehensive outcomes evalua- 

J SSaf ex^^lIncf in"th"*'^lTP°"?"^ t° enhance eJuM- 

IV " ^* IS in this spirit that I submit 

iLhPr pr"^-""''^^'*^V°"* reconraendation that the Board of 

Higher Education empanel an advisory committee is a necessary first 

^u'Ve ent^^thT'nroar.™'"' ^ reqSe^t thlt 

^Srova?o"f%he Boa?d°' • ' recommendation, for the 
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ATTACHMENT 1 



EXCERPT FROM STAFF MEMORANDUM 
•AN AGENDA FOR EDUCATIONAL EXCELLENCE- 
FEBRUARY 28, 1985 



Evaluation of Outcomes 



students are actuVn/learnlna i^ Liifl. '"tP"'.^*-'"" on whu 
were based on ths nr^TJL L I ^^''^Se- Their recommendations 
performance °s usSul'^'Ts a tSo? fn'/Vi^'V- """"e-t of 
for increasing student 1„vol»™ent • "iH^ expectations and 

iVt?^^r. j^ifetStiiri r °" H^'-'-"^' :s?es°:nd 

these cLslderatlons shUd „^^^^^^ concluded that 

Clear fo4ulat1on of pur„o«s a^d tfnJAt'""""""*' =3="=* 
thoughtful evaluat1o°n%?"S°udent/"''perfS™an«?"- 

** S°dj"u"l-fek'o7"°:^ '"'' '^""""""•■ 'e Examination, ... 
One aspect of "what students are actual Iv learnino i„ 

le^ned""S%nrcif5c"^H-^^'?^^. *^^*^'"9 °^ subject-matter 
iithnnnh .^^^^^Pl^"es at the college level 



What is i t that can be accurately measured ? 

The easiest variable to measure through normal testing 

methods is the acquisition and retention of particular facts 

and information considered fundamental to the field Facts 

and information, however, are not the -essence of 'what 

collegiate education has to offer. They are. of course. 

necessary raw materials, but they cannot be considered the 

desired product of the experience. Instead, the objlctivl 

of sound undergraduate education is to develop students' 

understanding of the coherence and internal structure of a 

discipline and of the Issues that constitute the substance 

of current work in the field, their ability to engage in the 

modes of thought and discourse used by its best 

practitioners, their appreciation of the complexity and 

structures in nature and society, and their 

aesthetic and moral sensibilities. Standardized tests 

fnI!^•^*^^""°^ measure these profound dimensions of personal 
and intellectual growth. 

2' What are the unintended side-effects? 

One of the dangers of testing what can be readily measured 
io. ! r^^^ f teachers to help their students prepare 
for the tests. But to concentrate on teaching what can be 
(or IS) measured using standardized tests would subvert much 
iLa^ 1' ^^^^ collegiate education. In effect, it 
might produce "results'-^by trivializing the objectives. 

Closely related to this concern is the danger that 
introduction of uniform tests of content in subject 
disciplines will lead to a standardization of curricula and 
a loss in the diversity of approach and opinion to which 
lllt^ll^ ^""^ .e*P°"d- Standardization of content might be 
seen as a gain by those whose orthodoxy prevailed, but it 
H?c!n constitute a disincentive to originality and 

djssent the dual responsibilities of the academic vocation 
at Its best. 

3. Is the information gained worth the price? 

To achieve statistical accuracy in measuring any but the 
most trivial aspects of knowledge gained from college 
programs, if possible, at all. would require either 
short-answer tests of great length or essay tests that would 
be exceedingly expensive to score in a reliable manner. The 
direct costs of developing and administering such an 
assessment program, when added to its potential for 
producing unwanted side-effects, make it unattractive if a 
better method can be found. »c ir a 
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*" assessment of knowledge gained from 
'*t! r^r!S:f»„ °^J"«ruction 1s that re'co^nded bT 



A 



adoption of policies at each college requiring 
adroinistratlon of cumulative tests and 
comprehensive final examinations in all 
appropriate courses; 

local development of program-specific graduation 
requirements by the faculty of each degree 
program. Including comprehensive essay 
examinations in each major field of 
concentration on which students would be 
required to perform satisfactorily in order to 
qualify for the degree; 

to which a third should be added: 

conduct of a systematic study of grading 
practices at each institution in order to 
Identify and correct cases of "grade inflation" 
that would render meaningless the results of 
local testing systems. 

^- Assessment o f Academic Proficiencies at the Sophomore Level 

In conjunction with the" points just outlined, the Task Force 
on Pre-College Preparation also reconmended: 

development and administration of a statewide 
test In writing and mathematics at the end of 
the sophomore year, a test which students would 
be required to pass in order to qualify for 
upper division courses. 

JtJUS^SS^^^f! difficulties in constructing any 

standardized testing program, both substantive and political 
in nature, this recommendation could be implemented in a 
constructive manner. The difference between testing 

*ocJ?n 9^"s^Tc academic skills and 

testing their mastery of specific domains of knowledge lies 
precisely in the difference between an ability that has 
general applicability and the possession of specific 
information. The generic skills of verbal and mathematical 
literacy, of critical thinking and problem-solving, are the 
lingua franca of higher education, the basic tools of 
discourse that permit access to the realms of thought to 
which collegiate education provides a guide; they are common 
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to all intellectual endeavor. In contrast, the specific 
factual knowledge that accompanies understanding of a 
discipline can vary widely. 

In designing an assessment of generic academic 
proficiencies, many issues of purpose and content will have 
to be considered and resolved. An essential ingredient will 
be the willingness of the community of institutions to 
participate in the design of the program and their 
confidence in the results. 

To this end, the Department of Higher Education should 
empanel a commissfon of knowledgeable faculty members from 
all sectors of the higher education system to study 
alternative methods of assessing the verbal and quantitative 
proficiencies of college students toward the end of the 
sophomore year, and report to the Board of Higher Education 
on the feasibility of instituting such a program. 

The implementation of any sophomore level assessment system 
should proceed in close coordination with the steps to be 
taken by the Department and Board of Education toward 
instituting a high school graduation test at the eleventh 
grade level. 

Teaching a nd Evaluation of "Critical Thin king" and 
"Hroblem-Solving" SkfTT? ^ 

In recent years the Ba'sic Skills Council has studied the 
data emanating from its testing program in order to identify 
instructional implications that could lead to improvement in 
students' proficiencies. On the basis of this analysis and 
the cumulative teaching experience of its members, the 
Council concluded that the development of students' 
"critical thinking" and "problem-solving" skills ought to be 
the principal underlying goal of educational programs at all 
levels. 

By the term "critical thinking" the Council means such high 
order skills as the following: 

the ability to extract, record, and reorder 
relevant information from' reading material; 

the ability to draw inferences that go beyond 
the literal meaning of a text; 

the ability to distinguish fact from opinion, 
and the ability to evaluate and use factual 
evidence; 
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the ability to construct a logically coherent 
argument and to present it in the form of 
expository wrning (as opposed to pure 
narration, description, or expression of 
feelings); 

the ability to understand nuances of meaning and 
to apply them appropriately in order to draw 
fine distinctions; 

the ability to modify an opinion in the light of 
new evidence and argument. 

the ability to understand what a problem 
consists of and what type of answer would 
constitute a solution; 

the ability to recognize that a proposed 
solution is nonsensical or of an inappropriate 
order of magnitude; 

the ability to translate a concrete problem into 
the abstract symbolic language of mathematics: 

the ability_to bring appropriate problem-solvma 
techniques to bear on new situations, i.e. the' 
ability to go beyond the rote application of 
algorithms to stock problems; 

the ability to carry out calculations and 
algebraic manipulations requiring multiple steps 
and the use of techniques from more than one 
region of the mathematical realm. 

To explore these ideas further, the Council established an 

riVnl'i? i?"?^ }? y^'"^^ 9*^^ "Task Force on 

to serve on it were faculty from an 
array of disciplines that bear on the question of teaching 
and learning critical thinking and problem-solving skills 
As the organizing principle for its' work, the group has 
formulated an agenda that can best be summarized in terms of 
four sets of questions: 

1. What kinds of generic "thinking skills" should 
students have when they enter college? Are certain 
Skins particularly important to academic success as 
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opposed to other kinds of human endeavor? Can one 
Identify skills individually or in combi-nation that 
are important in all or many content areas? 

2. Which generic "thinking skills" do students typically 
possess upon college entry, and which do they 
typically lack? ^ 

^' StJi J",^^ ^° improve students' generic 

i^JnilS^h'^^^^S^" 'i^-^'.l ""Id significantly 

Improved through explicit instruction: At what ages 

^thnH^c? Skills taught most effectively, and by what 
methods? Should generic skills be taught through 
deliberate application within the traditional 
curricular subjects, or should they be taught 
explicitly and separately from the traditional 
curriculum? 

4. What additional research on generic -thinking skills- 
should be undertaken? ^ 

When its work is complete (perhaps in another year). 

flrJ ^y^' *° '"^'^^ "^^'"ds of recommendations: 

* r '* . ^".^^l^ate ways to enhance the development of 
llrnnH K """^L*'!^!: ^^'^"king and prcMem-solving skills; and 
o? ?;2dPnt. '^^"V-^^- P''°'"^'s\"9 techn iques for the essessmeSt 
of students' proficiencies in these skills. 

Assessment of Students '"Progress Toward a Degree 

JnH^rllo*^^'"^*',^''"* equally important, set of outcome 
indicators would measure various aspects of students' 
progress toward earning a degree. Whereas direct 
^hf^^TiW r f"°'^ledge and skills proficiencies describe 
the intellectual progress of individuals and groups of 
rJn2?! " institutional retention rates. 

nu!w2 a.tte'npted and credits earned by students, and their 
grade point averages describe students' progress toward 
>r"?JSnn ^ credential that they will present to the world 
..t large as an affirmation of their educational 
achievement. 

No less than testing and the use of test results, the 
creation c\nd interpretation of information on progress 
toward a degree requires a subtle understanding of the 
complexities that contribute to a particular measurable 
outcome. 
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No comprehensive study on retention in New Jersey's colleges 
and universities has ever been conducted by the Department 
of Higher Education, though some individual institutions 
have analyzed their own experience of attrition and 
retention. The principal conceptual obstacle to a 
systemwide study that will yield a reasonably accurate 
picture is the difficulty of collecting data that reflect 
the essential distinctions among various kinds of attrition, 
some of which may actually indicate success rather than 
failure, and sone of which may be entirely neutral. The 
effort must be made, therefore, to think through the maze of 
possibilities, and to analyze how much of the gross 
attrition rate is undesirable. 

Measures of students' behavior such as credits attempted, 
credits earned, and grade point averages are simpler to 
handle than retention because they do not depend so directly 
on the ability to collect data on motivations and intentions 
that are difficult to quantify. Nevertheless, since grading 
philosophies and standards vary greatly among institutions, 
and since grades are closely related to definitions of 
credits earned, data from different institutions are 
exceedingly difficult to interpret outside the local context. 

Despite the difficulties of measurement, progress toward a 
degree is a fundamental outcome that is amenable to study 
and that should be^ better understood. The Department 
should, therefore, consult with the institutions in New 
Jersey on the conceptual structure of a meaningful study and 
methods of collecting the necessary information. An 
important source of guidance with respect to such an effort 
would be the National Center for Education Statistics, which 
is planning a longitudinal study of persistence, transfer, 
withdrawal, student characteristics, and academic 
performance; the initial collection of data is proposed for 
19oo. 

Assessment of Post-Graduation Activity 

Whether or not a student's education leads to further 
"opportunities" and "success in life" is perhaps the 
ultimate measure of outcomes. The profound questions 
suggested by a liberal interpretation of "opportunity" and 
'success" are. obviously, innumerable; they could lead to a 
rich variety of sociological studies. For the immediate 
purpose of assessment, however, it is only realistic to 
limit the scope of Inquiry to those effects that can be 
measured by colleges and universities and the incidence of 
which can be inferred to have a causal connection to 
educational programs. Two measures of such effects are the 
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degree to which students pursue further education or 
successfully enter upon the initial stage of a career after 
completing their dfegree programs. The latter may. in turn. 
fLTnln"tf*^ categories of career placement ' 

i??Sn«^f!^ and successful passing of professional 
licensing examinations. 

As always, the collection of accurate information about 
these categories of behavior Is difficult. Causal 
relationships may seem self-evident but often turn out to be 
tenuous. Methodologies vary, but they all run into the 
difficulty of winning the cooperation and participation of 
former students In all the categories needed to present a 
representative description of post-graduation activities. 
And due consideration must be given to the widely varying 
missions of different institutions and purposes of 
Individual programs in designing an assessment of outcomes. 

Once again, however, information about these relationships 
IS essential to any overall understanding of the 
contributions higher education makes both to individual 

^in^?2^hr'^^*° ^^'•S^* The Department 

should, therefore, work with the colleges and universities 

lmi!n! ; T*^/ °J l":veying a representative sample of 
college graduates (at all degree levels) regarding their 
subsequent activities. Such a system might well be based 
upon the various questionnaires currently in use. A 
complementary analyticaj framework should be devised In such 
a way that relationships among a multitude of variables can 
be described and that reports can reflect the diversity of 
behaviors evident in the choices people actually make. 

Reporting on Student Performance 

The information generated from any new assessment program 
J^hL fH"^ disseminated to appropriate users in 

order to maximize Its effect. Information need not be new 
however, to be useful. Much of the information that already 
exists about students' performance could be put to better 
use If It were properly analyzed and distributed. An 
instructive example Is the system of reporting the scores 
earned by entering college students on the Basic Skills Test 
to the high schools from which they graduated. Much other 
information that is presently available on students' grades 
and progress toward successful completion of programs could 
be shared In a similar way with Institutions that have an 
interest in the subsequent performance of their students 



- 14 - 



Mft^^U^g"^^^ Should Plan a progra. of 

"ioJ?|%r,-J^,;^,^^ to sending h1,H 

results .but also detailed w"'^ Skills Test 
performance through the fir.tT^^'^" °" ^^eir 
college. . ^"^ ^^i^st two years of 

likewise, four waa« 

sending two-^ei^*"^ i^HYtT, ^'"""<' Provide 
onn.e peS:^- tSelj^l^.er 

-tuuti/n; ro'Jv^TnnrVa'sV. °^ twS^rs"^?" 

g?tlt; o"utrer"ir:S;T„n^°'.TH,"""' seve^a, 

^nJj'-K^!-*' '■""e'- than c^latl h.'^?*" should offer 
"ntr,5.te to t.e assess^^o? pXV^;>fec^^^^es^'<' 
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State of New Jersey 

Board of Higher Education 

225 West State Street 
Trenton, New Jersey oaeao 

(609/ 292*9879 

MEMORANDU M 
June 12, 1985 

TO: Members, Board of Higher Education 

FROM: Martin Freedman, Chairman 
Academic Affairs Committee 

SUBJECT: College Outcomes Evaluation Program 



The Academic Affairs Committee met on May 28 to consider a colleae 
S i iTVom S.°p^^°^^'7^^ '''''''' ^'^ attached me^I'?? 

art?o^^?l^:??ac;Vd^ ' fortJe'%?a?5?s^ 

The proposed program, which is to be thoroughly studied bv an 
ftriV/ """^"ee. would create a comprehensive system for evafuat^ng 
the effectiveness of institutions of higher education in the state 
The elements selected for evaluation fall into the areas of students' 
progress and post-graduation activities, and institutions' impact on 
the surrounding community and the broader society. The testinS of 
students in the sophomore year of college for col lege -level academic 

SSe 'advTsor'v InJ^^'^^'^V "T"^"* °^ progr'^. Tn th" case 
che advisory committee will be charged to recommend the content 
design, and mode of implementing the'assessment. The other eWts 
outlined seem necessary for a sound and comprehensive evaluat o?- in 
these cases the committee will be asked to study the feasibilitv of 

fmoL'inr't"- ^ P^^f^^^^d mo^de of evaluation 

implementation. Ihe committee will, of course, be at libertv to 
recommend additional elements to the program. Wisely there w^l be 
room for institutional discretion in ^selection of sJe of the 
"outcomes" to '.e evaluated, instruments to be utilized ^d utiliza 
tion of the data produced by the program. "'-^'izeo, ana utiiiza- 
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The focus on -outcomes" is an interesting approach, as it directs 

JjJr!liJ?Jh*J^,*^\";J''^^ question of what the institutions actua.ly 
acccmplish through their various activities. This is to be done 
whenever feasible, by measuring "value added," or the difference that 
an institution's intervention has made. Thus the inf^rmatio^ geJe! 
rated by the program would convey a sense of improvement in o^rfor- 
mance as well as absolute levels of achievement. As we know, this is 
especially important given the diversity of student bodies in New 
^^^"?:%.^"n:tutions. A major purpose of the program wi?l Se to 
support institutional efforts toward excellence within the context of 
each institution's mission and goals. coniexi ot 

Overall, we believe that the program as proposed meets the con- 
in?in°!r J-^ institutions, the Chancellor, and the Board, for further- 
ing educational excellence and demonstrating the value of colleqiate 
education and institutions. The Committee 'strongly urges you to ap- 
prove this initiative by adopting the attached resolution. ^ 

Attachment 




23 




Slvte or N'env Jehsey 
Sx4ns BojlRd or Higher Edccatiox 

A BESOLDTIOH ESTABLISHING AN ADVISORY COMMITTEE 
TO EECOMMERD THE DESICR AND MODE OF IMPLEMENTATION OF A 
CQMPHEHENSIVE COLLEGE OUTCOMES EVALUATION PROGRAM 



WHEREAS: 



The Board of Higher Education wishes to maintain 
public confidence in, and ensure continued support 
and funding for, the New Jersey system of higher 
education by demonstrating the value of its pro- 
grams; and ^ 



WHEREAS: 



The Board has discussed, and at its meeting on 
March 15. 1985, expressed interest in the develop- 
ment of a comprehensive program to measure outcomes 
of higher education; and 



WHEREAS: 



WHEREAS: 



WHEREAS; 



The Joint Statewide Task Force on Pre-College 
Preparation recommended the development of selected 
r!«r"K J^'^""*, measures, including a statewide 
test battery for college sophomores, and the 
development of specific graduation requirements; and 

The Board, in its resolution of March 16. 1984 
ITcT'T^ institutions to review the Task 

Force s recommendation that a statewide test in 
verbal proficiencies be given to all students at 
the conclusion of the sophomore year in college; and 

Several national commissions and organizations 
including the Association of American Colleges in 
Its report "Integrity in the College Curriculum " 
and the National Institute of Education, in its 
report Involvement in Learning," have recommended 
the development of sound evaluation programs, both 
to maintain public confidence and to stimulate 
curricular improvements; and 
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The Board wishes to explore the development of a 
comprehensive evaluation program which is consis- 
tent with the joint principles of maintaining 
public confidence, nurturing institutional autonomy 
and individual diversity, and stimulating educa- 
tional excellence; now, therefore be it 



That the Board of Higher Educction supports the 
efforts of the colleges and universities to main- 
tain public confidence and high standards of 
learning and performance; and be it further 



That the Board of Higher Education agrees that a 
coirprehcnsive program to evaluate the outcomes of 
higher education can contribute to educational 
excellence in New Jersey and authorizes the 
Chancellor to proceed with instituting such a 
program; and be it further 



That an advisory committee, composed of students, 
faculty and administrators with the requisite know- 
ledge and expertise from all sectors of higher edu- 
cation in IJew Jersey, and members of the public who 
reflect the views of various constituencies of 
higher education, such as business, government, and 
nonprofit organizations, be appointed by the Board 
upon the Chancellor's nomination; and be it further 



That the said advisory committee be charged to 
study options and report to the Chancellor its 
recommendations on how best to design and institute 
a comprehensive system of evaluating the outcomes 
of higher education; and be it further 



That the evaluation system shall include an assess- 
ment of students' learning through the administra- 
tion of a test battery that measures proficiencies 
in writing, quantitative reasoning, critical think- 
ing, and any other areas appropriate for the 
evaluation of general college-level academic pro- 
ficiencies. The tests are to be taken toward the 
end of the sophomore year by all students attending 
public colleges and universities in New Jersey, and 
by students attending independent colleges and 
universities that choose to participate. The test 



battery shall be constructed so as to include the 
capacity to measure students' proficiencies in the 
basic skills after two years of college and to per- 
mit ccn^mrison with their basic skills pro- 
ficiencies at college entrance as originally 
measured by the New Jersey Callege Basic Skills 
Placement Test. The test battery may include some 
components conaaon to all institutions and others 
specific to individual institutions or groups of 
institutions; and be it further 



RESOLVED: That the said advisory committee be charged further: 

1. To consider the feasibility, design, and 
implementation of the following pqtgatial 
components of the outcomes evaluation system. 
In addition to the specific elements outlined 
below, the committee may recommend other 
elements as deemed appropriate and feasible. 

Student outcomes: 

institutional development of specific 
graduation requirements, including sat- 
isfactory performance on comprehensive 
examinations to be taken before gradua- 
tion in each major field of concen- 
ration; 

Qwvelopment of measurements of progress 
toward the degree and post-graduation 
activities relevant to institutional 
assessment; particular attention should 
be paid to minority retention and gradua- 
tion as a measure of institutional per- 
formance • 

Community-based/ society-wide outcomes : 

establishment, at the state level, of a 
matrix and guidelines for the evaluation 
of these outcomes; 

selection, at the local level, of appro- 
priate outcomes and development of 
instruments and practices for their 
evaluation; 
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— identification of models for the sound 
and objective evaluation of these out- 
comes and development of policies for 
their use on a system-wide basis. 

Feedback and information sharing: 

review of information systems presently 
in use and, if necessary, redesign so as 
to report the results of outcomes assess- 
ments in ways that cen be used construc- 
tively; 

implementation of a unit record system 
with standardized definitions and report- 
ing formats. 

To recommend ways of implementing the program, 
with specific reference to methods of evaluat- 
ing various distinct outcomes and identifica- 
tion of those elements that should be cen- 
tralized and those that should be designed and 
administered at the institutional level. 

To recommend ways of utilizing incentives to 
improve performance. 

To recommend an appropriate distribution of 
the costs of the program. 



June 21, 1985 
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